Users are increasingly utilizing electronic devices to obtain various types of information. For example, a user wanting to obtain information about a book can capture an image of the cover of the book and upload that image to a book identification service for analysis. In many cases, the cover image will be matched against a set of two-dimensional images including views of objects from a particular orientation. While books are typically relatively easy to match, as a book cover generally includes several features that enable the cover to be matched against a set of cover images, other objects are not as straightforward to match. For example, an object such as a men's dress shoe that is captured from the side might not have many distinctive features, and may appear primarily as a shaped black object in the image. In order to efficiently perform image matching for such an object, the object of interest is often first separated from the background portion of the image. Unfortunately, it can be difficult to separate an object that does not have many unique features that help to distinguish the object from the background. Accordingly, objects such as shoes can take longer to recognize, and the results can be less accurate on average than for objects such as books or media packaging.